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A goal of every designer is to create successful products for consumers. In 
creating a successful product, it is crucial for a designer to understand 
consumers’ perceptions of a product early in the design process. 
Nevertheless, design students lack the necessary data collection and user 
testing skills to support effective design decision-making. Consequently, 
their products might not be acceptable to the intended consumers and are 
thus likely to fail in the marketplace. For design students to acquire those 
skills, design curricula should incorporate statistical courses teaching the 
concepts of data and user testing. We addressed this challenge by developing 
an automated visual tool named DACADE, assisting design students to 
systematically collect and analyze data. This paper reports the theoretical 
implications discovered during the process from designing through to 
implementing and evaluating DACADE concerning the transfer of learning, 
the appropriateness of graphics used in a software tool, and user motivation 
in a learning environment. 
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I. INTRODUCTION 

A goal of every designer is to be successful in designing and developing products for his or her 
intended consumers. In order to be successful, as well as focusing on the objective requirements of a 
product, a designer should also consider consumers’ subjective requirements [1]. It is also important for a 
designer to understand consumers’ perception of a product early in the design process. This has increasingly 
been recognised as a serious concern for design students and novice designers who need to compete in the 
21st Century and become successful designers. One way for designers to ensure that consumers’ perceptions 
and ideas are taken into account is to gather data from the target audiences of their proposed products and 
then to analyse those data statistically. 

In our preliminary investigation, 20 design students and 10 design lecturers were interviewed; an 
email survey and archival search involving 51 universities that offer design courses in Asia, Australia, 
Europe and North America were carried out. The results suggested that both design students and lecturers 
were familiar only with some qualitative data collection methods but not with quantitative methods. The 
results also discovered that design students might benefit from a visual and interactive method to assist them 
in collecting and analyzing z 

As a remedy, we proposed DACADE: an automated visual tool for design students to 
systematically collect and analyze data. Prior to designing DACADE, we first reviewed the relevant 
theoretical frameworks to guide this research as summarized in the next section. 
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2. THEORETICAL IMPLICATIONS 

2.1. Schema Theory 

Schemata refer to the internal models of the world learned through experience and stored in the 
brain; they are primarily applied to predict future events but also to interpret situations as they occur [2]. Any 
new concept, idea or information can become meaningful when they can be related to existing knowledge 
that someone already possesses [3]. Schemata are exclusive to each individual [4]. There will be no two 
people storing exactly the same information in precisely the same fashion. This is evidenced by one of 
Bartlett’s experiments in which he found that individuals manipulated and retold a story they heard by 
applying their own unique interpretation and culture. It is also important to note that the more schemata one 
has, the better one can predict future behaviors [5]. Experienced drivers, for example, can drive any car, be it 
automatic or manual, better than novice drivers, because experienced drivers possess much more refined 
networks of schemata associated with driving. Schema theory would thus appear to be a convenient vehicle 
for explaining designers’ existing internal models for designing products. In relation to design students, 
whenever they are given the task of creating a new product, their existing knowledge, or schemata, related to 
product creation will be activated to guide the new product design. Designers may select only relevant 
information or schemata related to the product, as they may deem appropriate from their own perspective, 
understanding, interpretation and culture. In relation to the present research, similar to the example of drivers, 
it can also be inferred that young designers or design students are likely to have fewer schemata than design 
experts or experienced designers, thus limiting their ability to design more successful products. 

2.2. Mental Models 

The literature suggests that mental models are similar to the concept of schemata [2] in the sense 
that people use existing knowledge stored in the brain to interact or react with objects and events [6]. Design 
students may have existing knowledge relevant to user testing, statistical analyses, and interpretation to help 
them in making decisions for designing products. 

Mental models are not necessarily accurate; they are usually incomplete [7-8]. Eor instance, an 
individual with an accurate mental model of how an elevator works will understand that they only need to 
press the button once and wait, while people with an inaccurate or incorrect mental model might keep 
pressing the button until the lift doors open. This also suggests that mental models are based on individuals’ 
personal experiences, beliefs and knowledge. Thus, everyone’s experiences differ from everyone else’s, 
another criterion similar to Schema Theory. Eor instance, designers are likely to have some knowledge, or 
even to possess complete mental models, of sketching and drawing using an AUTOCAD drawing tool, 
whereas lawyers, who have little or no knowledge of sketching and drawing, would not. In contrast, a 
designer may have no knowledge of what legal proceedings in court are all about unless they have some 
experience of attending one. However, knowledge and experience really depend on the nature and task and 
the way individuals interact with their environment [9]. Individuals’ experience of grocery shopping is 
another example of such interactions. People usually know where to find bread and milk on the shelves 
regardless of which shops they go to; they can also search the signage provided in a shop. This is because the 
mental model of shopping for bread and milk is already stored in their minds. By contrast, the process of 
buying or renting a house may differ according to individuals’ experiences, such as when dealing with fussy 
landlords or agents. With respect to this research, design students coming from different cultural and 
educational backgrounds as well as varying personal experiences may have different mental models or no 
knowledge of data collection, analysis or even what data sampling should be applied when designing new 
products. Therefore, their understanding, perception and acceptance level of new information or knowledge 
will differ. Mental models provide another valuable guideline for this present research. 

2.3. Cognitive Complexity Theory 

A second model related to designing user interface is Cognitive Complexity Theory (CCT) 
introduced by Kieras and Poison [10] and used in Human-Computer Interaction (HCI) in the mid-1980s, also 
in the era of command-based and menu-driven interfaces. While GOMS aimed at the structure and content of 
the knowledge, CCT represents the amount of knowledge users need to successfully complete their tasks. 
That knowledge is represented by a set of system rules. In that sense, CCT may be regarded as an extension 
of GOMS [11]. Elaborating further. The GOMS framework proposed by Card, Moran and Newell (23,24) for 
user interface design comprises a set of procedures including goals, operators, methods and selection rules 
used to predict the efficiency of a computer system or comparing two or more systems by calculating the 
time it will take for a user to complete a given task [25]. 

The main concern of CCT is to ensure that a new computer system is ‘easy to learn’ and ‘easy to 
use’ [11]. CCT suggests providing users with procedural knowledge or ‘how-to’ knowledge [12] when using 
a tool to execute a task. One could thus offer several techniques for collecting data, leaving it to the user to 
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choose the best technique for the particular product design context. Once a technique has been selected, the 
user should then be able to follow a step-by-step procedure; for example, to create a survey, taking them 
through the sub-tasks of defining a study, deciding upon the sample size, defining demographic questions, 
entering a series of questions, saving and running the study with potential respondents. 

The procedural knowledge is therefore useful for novices. To the extent that several data collection 
methods are offered in a single tool, this renders the tool very flexible. It would enable users to compare 
results obtained through different data-collection and analysis techniques. Nevertheless, flexibility would 
also increase the amount of learning for novices to use the application effectively and efficiently. However, if 
a flexible application offers too many ways to complete tasks, that could prove too complex for novices with 
no knowledge about user testing and data analysis. The preliminary data gathered in our earlier investigations 
[13], showed clearly that design students had no working knowledge of statistics and hence no recollection of 
the steps and actions necessary for calculating, for example, the arithmetic mean or the median of a dataset. It 
therefore seems essential to provide adequate step-by-step information in the proposed tool to help them 
comprehend and eventually anticipate the steps needed when analyzing data statistically. Hence, this study 
follows CCT in the sense that it is essential to provide procedural knowledge or step-by-step instruction as 
highlighted in Scripts for a design student to accomplish a task in the proposed tool. 

2.4. Cognitive Theory of Multimedia Learning 

CTML [14] promotes the use of multimedia elements such as graphics, words, sounds, animation, 
video and gesture to foster learning. Among others, CTML suggests that people possess active processing 
capacity. This implies that learners will attempt to make sense of information when they pay attention to the 
relevant material by selecting, organizing and integrating it with their prior knowledge [14]-[16]. 

While there is some evidence to support the claim that presenting information in several techniques, 
such as the use of graphics or other elements, aids learning [17], other research suggests that this is not 
normally the case. For example, [18] carried out an experiment to investigate the validity of CTML 
specifically on the multimedia, spatial contiguity and coherence principles to facilitate learning by adding 
external graphic representations on paper-based and electronic learning materials among first-year students at 
a university. In CTML, multimedia principle focuses on the use of multimedia elements such as graphic, text, 
video and animation to facilitate learning, spatial contiguity principle emphasizes placing words and pictures 
on the same page/screen, and coherence principle highlights the need to exclude extraneous sounds, words 
and pictures to facilitate learning [19]. The study took special care in producing the graphical representation 
for both printed and animation materials by consulting at least 20 fourth-year students to find the best 
graphics. The materials given were considered to be difficult and new to the first-year students because they 
had no prior knowledge of the materials. They were given a pre-test to determine their prior knowledge, 
training with the relevant materials, and post-test questions for retention and transfer tests. Students learned 
with no graphics representation; with illustrations that were not spatially integrated; and with animations and 
sound but no narration, performed better. The study concluded that instructional designers could not simply 
apply CTML in designing instructional materials. However, this study suggests that instructional designers 
need to take extra care when choosing any graphical representation for a learning material relevant to 
individuals’ differences. This was a crucial issue that was not addressed in this experiment and could lead to 
different results. This finding appears comparable to Cook [20] in the sense that suitable design and 
consideration between different learners need to be taken into account due to different prior knowledge, 
experience, skills and preference among learners. 

2.5. Cognitive Load Theory (CLT) 

Another model relevant to this research is known as Cognitive Load Theory (CLT), proposed by 
Sweller and his colleagues in 1980s. CLT [21] suggests that a major contributor to cognitive load is the 
quantity of elements that need to be attended to by learners. "Elements" means the content of information 
presented to an individual [21]. Complex information, such as mathematical calculations, can be difficult to 
learn-especially for design students who are not accustomed to rely on numbers in any way, as the 
preliminary data showed. Chunking information into elements or smaller sections is recommended to 
promote learning considering the lack of skills and knowledge of user testing among design students. Hence, 
the tool should be organized into several sections. 

CLT was selected as the most relevant theory to guide the design and development of this tool due 
to several reasons with respect to design students’ prior knowledge to run user test and perform statistical 
analysis. This theory provides useful guidelines to be considered. Firstly, it provides a simple tool that allows 
design students to easily understand and appreciate the tool. Secondly, designing a tool in such a way that it 
can be divided into smaller sections, frees up more space in their working memory to process and use the tool 
effectively. Thirdly, using different instructional techniques to present the information in the new tool; in this 
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case, visual techniques for data collection and analysis, are highly relevant to designers generally, and design 
students particularly. 

This section summarizes briefly the theoretical frameworks underpinning this research. Considering 
the limited knowledge of user testing and statistical analysis among design students, it is, therefore, necessary 
to design a simple and easy-to-use software tool for data collection and analysis, presented in different 
instructional techniques and smaller sections to foster learning. The following section describes the methods 
used for conducting this research. 


3. METHODS 

Paper prototypes of DACADE were prepared and evaluated in two formative usability tests to 
ensure that DACADE would not contain any usability flaws [13]. The prototypes were divided into four 
modules with four tasks, namely (1) designing a new study, (2) editing an existing study, (3) collecting data 
(i.e. running the study) and (4) conducting simple descriptive statistical analyses. The prototypes were 
evaluated iteratively with 11 design students at one design institution in the formative usability tests to satisfy 
the pre-determined usability goals. Based on the results in the first usability test; a tutorial was added and 
tested in the second usability test to aid design students’ DACADE usage, as described in our earlier 
published work [13]. Upon implementation, a User Acceptance Test was conducted with 13 new participants 
to determine the level of user satisfaction, perceived usability and aesthetic appeal of DACADE among 
design students [22]. Results indicated that, despite the presence of the tutorial, the design students tested 
were still unable to grasp the necessary understanding and acquire the essential skill of collecting and 
analyzing data systematically. Based on the theories reviewed earlier, this study revealed several theoretical 
implications discovered during the design, implementation and evaluation of DACADE that could be 
beneficial for future research as briefly discussed in the next section. 


4. THEORETICAL IMPLICATIONS 

4.1. Transfer of Learning 

This study followed the relevant guidelines provided by the theoretical frameworks discussed 
earlier. Eor example, dividing the task into the sections of New Study, Edit Study, Collect Data, and Run 
Analyses as recommended by CLT. Secondly, the study followed the principle of CCT, where the tasks were 
organized in steps, enabling users to simply follow the prompts in completing tasks. Nonetheless, it still 
seems insufficient for the design students to be able to grasp the gist of using DACADE and they did not 
perform well in the task. One possible reason for this could be that not enough attention was paid to the given 
tool and tasks. The findings suggest that the participants in the earlier usability studies [13], ]22] did not 
know or grasp the notion of collecting data from several consumers prior to deciding on a particular design. 
Taking shopping for a mobile phone as an example that participants would be likely to have experienced, one 
would investigate several plan and phone options in different physical and online shops, looking for good 
deals before settling on a decision. That example suggests that they actually would have some existing 
knowledge of sampling and comparing different possible solutions. This would indicate that they do 
understand the idea of data, but that they did not manage to transfer that knowledge to the present context. 
That is, participants appeared to have neglected to activate the relevant mental model enabling them to 
transfer their knowledge to a different type of problem. This problem of transfer has not been addressed by 
the theories reviewed in this study. 

4.2. Appropriateness 

The use of bananas as a graphical representation in our earlier study demonstrating arithmetical 
central tendencies was not ideal for the design student population [13]. It appeared that they simply focused 
on the salient features of the object and interpreted that as having sexual connotations, rather than 
recognizing them as representing the underlying meaning of descriptive statistics. The extent to which the 
apparent inappropriateness of the images initially selected may have affected participants’ performance 
cannot be determined post hoc, but it may have played an important role. No theories were found that 
discussed the appropriateness of symbols selected to support the tasks and the audience. 

4.3. Motivation 

Another important finding of this research is that design students were hesitating to read the tutorial 
and the tasks, basically saying that they preferred illustrations, animations, and anything that would be more 
entertaining than text. Indeed, it took the researcher several attempts to encourage them to read before they 
would do it. Even so, several issues raised suggested that they did not pay as close attention to the 
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explanations, examples, and even exercises as they needed so as successfully to complete all of the tasks in 
the tutorial as well as when using the tool. This suggests that learners in general, and design students in 
particular, at least in the present situation, need to be extrinsically motivated to learn. This important issue is 
not addressed in any of the learning-related theories reviewed in this paper. 


5. CONCLUSION 

This paper revealed several important theoretical implications related to learning theories in the 
process of developing DACADE. The implications related to the transfer of learning, the appropriateness of 
graphics used in a software tool, and user motivation in a learning environment. It is hoped that the 
implications discussed in this paper could serve as a as a guideline for future research in user-interface design 
as well as to foster learning. 
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